Cache memory is one of the critical elements in computer processors for achieving good performance on the processors. Generally, a cache is a smaller, faster memory used by a central processing unit of a computer to reduce the average time to access its data or instructions. The cache typically stores copies of the data from the most frequently used main memory locations. The fundamental idea of cache organization is that by keeping the most frequently accessed instructions and data in the fast cache memory, the average memory access time will approach the access time of the cache. A cache miss is costly because the data must then be fetched from a higher-level cache, main memory, or potentially another processor's cache on a multiprocessor, which incurs a delay because accessing the other memory is slower than accessing the cache memory. Thus, maximizing the cache's hit rate is one of the important factors in achieving good performance.
The configuration of a cache is a direct factor affecting the cache hit or miss rate for execution entities such as processes and threads that execute on a processor. It is often the case that different processes and threads require different cache configurations in order to maximize that process or thread's cache hit rate. In conventional cache design, however, many of the characteristics or attributes of the cache are set at design time. This is true even though programs have dynamic requirements for use of the cache. Accordingly what is needed is a method for allowing the hardware to reconfigure cache organization, for instance, based on dynamic or otherwise tracked information.
Various requirements that change over time sometimes can be known by the operating system or software. Also, the software or the operating system can determine different access patterns among different applications or threads. In other cases the operating system can proactively know the behavior of a given application. Therefore, it is also desirable to provide a mechanisms so that the software or operating system or the like can provide the information relating to the dynamic requirements of different programs, processes, threads, or the like, thereby allowing the hardware to configure the cache to suit the needs of the given application